Papers by Cyprien de Masson d’Autume
A Systematic Investigation of Commonsense Knowledge in Large Language Models (2022.emnlp-main)
Copied to clipboard
Xiang Lorraine Li, Adhiguna Kuncoro, Jordan Hoffmann, Cyprien de Masson d’Autume, Phil Blunsom, Aida Nematzadeh
| Challenge: | Recent large language models (LMs) have shown impressive performance on many NLP tasks under the zero-shot and few-shot setup. |
| Approach: | They conduct a systematic and rigorous zero-shot and few-shot commonsense evaluation of large pre-trained language models to better understand their ability to capture commonsensical knowledge. |
| Outcome: | The proposed model can exploit surface cues and annotation artefacts without task-specific supervision and is insufficient to achieve human-level commonsense performance. |
Adaptive Semiparametric Language Models (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing language models that use a large parametric neural network with episodic memory are not efficient. |
| Approach: | They propose a language model that combines a large parametric neural network with a non-parametric episodic memory component in an integrated architecture. |
| Outcome: | The proposed model can predict local context, short-term memory, or long-term memories on an ad hoc basis depending on the context. |